
PATENT APPLICATION 
Attorney Docket No. D/99423Q 



APPLICATION FOR 
UNITED STATES LETTERS PATENT 

To Whom It May Concern: 

Be it known that I, Dennis L. Venable, citizen of the United States of 
America and residing at Marion and State of New York, have invented an 
improved: 



DETECTING OVERLAPPING IMAGES IN AN AUTOMATIC IMAGE 
SEGMENTATION DEVICE WITH THE PRESENCE OF SEVERE BLEEDING 




DETECTING OVERLAPPING IMAGES IN AN AUTOMATIC IMAGE 
SEGMENTATION DEVICE WfTH THE PRESENCE OF SEVERE BLEEDING 

This invention relates generally to detection of a particular 
5 images of several documents placed on the platen of an image input device in 

the presence of severe background bleeding, and more particularly to the 

automatic detection of the overlap of a series of images placed on the platen 

using the technologies of joining orphaned images, linear regression 

algorithms and collinearity detectors. 
10 Cross reference is made to the following related applications are 

hereby incorporated by reference for their teachings: 

"METHOD AND APPARATUS FOR IDENTIFYING A PLURALITY 

OF SUB-IMAGES IN AN INPUT IMAGE," Dennis L. Venable et al., Application 

No. 08/786,538, filed January 21 st , 1997; 
15 "STRUCTURED IMAGE (SI) EDITOR AND METHOD FOR 

EDITING STRUCTURED IMAGES," Michael R. Campanelli et al., Application 

No. 08/338,856, filed November 14, 1994; and 

"LOCATING THE POSITION AND ORIENTATION OF MULTIPLE 

OBJECTS WITH A SMART PLATEN," Dennis L. Venable., Application No. 
20 08/785,109, filed January 21 st , 1997. 

"AUTOMATIC IMAGE SEGMENTATION IN THE PRESENCE OF 

SEVERE BACKGROUND BLEEDING," Dennis L. Venable, Application No. 

D/99423, filed 1999. 
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BACKGROUND AND SUMMARY OF THE INVENTION 

To improve the ease of use of scanning systems, in particular 
digital copiers and scanners, it is desirable to automatically detect and 

5 segment scanned objects (e.g., a hard copy document) on the platen of the 
input scanner. To facilitate this automated detection and segmentation, it is 
necessary to identify the position, shape and rotation angle of the object. Such 
a capability can enhance productivity by decreasing the time required for 
scanning documents, enable multiple images to be efficiently scanned and 

10 reduce the requirement for accurate presentation placed upon document 
feeding mechanisms. 

The present invention is directed to a system intended to 
automatically determine the boundaries and overlap of multiple objects within a 
scanned image of a series of documents. The present invention combines a 

15 number of graphics and image processing techniques into an automated, 
system that provides productivity enhancement for digital copiers and scanning 
systems. 

The present invention accomplishes these objectives by: 1) 
locating each of the objects within an input image; 2) modeling the shape of the 

20 identified object (e.g., rectangle); and 3) positioning the object in a digital 
image at a predefined locations. 

Heretofore, a number of patents and publications have disclosed 
segmentation and detection, the relevant portions of which may be briefly 
summarized as follows: 

25 US-A-5,485,568 to Venable et al., issued January 16, 1996, and 

hereby incorporated by reference, discloses a method and apparatus for 
representing a complex color raster image as a collection of objects in a 
structured image format - a hierarchical, device-independent format. A 
structured image document, generated using the techniques described by 

30 Venable, is a representation of data that may be rendered into a raster image. 
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The data includes simple raster images as well as a hierarchical collection of 
sub-objects and raster processing operations. The possible data types for 
objects in the structured image include a raster image, text, graphics, image 
processing description, and files containing multiple image representations 
5 US-A-5,528,387 to Kelly et al., issued June 18, 1996, teaches 

electronic image registration in a scanner. In particular, the edge data of a 
document is detected and skew angle calculated. The image is then rotated 
based upon the skew angle and non-image areas are filled using an image 
generation feature. 

io US-A-4,922,350 to Rombola et al., issued May 1, 1990, discloses 

a two-pass scanning apparatus for detecting the size and position of an 
original document on a scanner platen. Image signals captured on a first scan 
are employed to determine boundaries and a best-fit magnification so that the 
image may be fit to a recording sheet using image signals generated on a 

15 subsequent scanning pass. 

US-A-5,253,765 to Moorehead et al, issued Oct. 19, 1993, 
teaches a system for sorting randomly sized objects (e.g., mushrooms). 
Invariant moments are employed, utilizing the complete pixel information for all 
pixels within the border of a captured image, to extract information about the 

20 mushroom size and orientation. 

US-A-5,220,398 to Horn et al. teaches an analog VLSI microchip 
that uses moments to determine the position and orientation of an object in a 
scene. 

In "Invariant Fitting of Planar Objects by Primitives," published in 
25 1996 IEEE Proceedings oflCPR '96, pp. 508-512 Voss et al. teach a method of 
pattern recognition using primitives such as triangles, rectangles, circles 
ellipses, super-quadratics, etc. The authors further describe a technique for 
describing the primitives using moments in a normalized manner; resulting in a 
decrease in the numerical effort. 
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In "MANAGING AND REPRESENTING IMAGE WORKFLOW IN 



PREPRESS APPLICATIONS", Technical Association of the Graphic Arts 
(TAGA) Vol. 1, 1995 Proceedings pp. 373-385, hereby incorporated by 
reference for its teachings, Venable et al. teach the use of structured images to 

5 manage workflow. 

In accordance with the present invention, there is provided a method 
fpr processing multiple digital images using an imaging input device so as to reduce 
bleeding of edges of the multiple digital images by determining the boundaries of 
each of the digital images, comprising: 

io detecting a boundary of a first image; 



second images wherein the third image contains at least said first and second 
images and represents a depiction of said first and second images without an 
overlap between said first and second images. 

A method for processing multiple digital images using an imaging input 
20 device so as to reduce bleeding of contour edges of the multiple digital images by 
generating an object defined by contour edges of particular sets of the multiple 
digital images, comprising: 



and second objects; 

calculating the overlap between the set of edges of the first and 
second objects; and 

modeling a third object by ascertaining the calculated overlap of 
30 the first and second objects wherein the third object contains at least said first 
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detecting a boundary of a second image; 

determining an overlap between the detected boundaries of the first 
image and second images; 

calculating the overlap between the first and second images; and 
modeling a third image from the calculated overlap of the first and 
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detecting a set of edges of a first object; 
detecting a set of edges of a second object; 

determining an overlap between the detected set of edges of the first 
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and second objects without an overlap of the set of edges of the said first and 
second objects. 

The techniques described herein enable a user to expediently 
scan a document or other object, automatically recognizing the shape of the 
5 document within the digitized image, and composing a digital document 
incorporating the object. 

The techniques described above are advantageous because they 
improve the efficiency of a scanning process, in the presence of severe 
bleeding. 

10 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is an illustration of an exemplary digital color printing 
system serving as an embodiment for the present invention; 

Figure 2 is a block diagram of the various components 
comprising the system of Figure 1 ; 
15 Figure 3 is an illustration of the placement of a document object 

on a platen of the scanner depicted in Figure 1 ; 

Figure 4 is a flowchart illustrating the general processing steps 
carried out on the system of Figures 1 and 2 in accordance with the present 
invention; 

20 Figure 5 is a detailed flowchart illustrating a series of processing 

steps carried out to initially determine the "seed" of the images; 

Figure 6 is a more detailed flowchart showing the steps of 
determining the "seed" of the images. 

Figures 7 is a detailed flow chart illustrating series of processing 
25 steps carried out in accordance with the present invention; 

Figure 8 is an illustration of line estimation in accordance with the 
present invention. 

Figure 9 is an illustration of a bin categorization in accordance 
with the present invention. 
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Figures 10A, 10B, and 10C are illustrations of a bin removal 
operation in accordance with the present invention. 

Figure 11 is an illustration of a midpoint averaging between bins 
in accordance with the present invention. 
5 Figure 12 is a detailed illustration of a step of the flowchart of 

Figure 6 in accordance with the present invention. 

Figure 13 is a detailed illustration of a step of the flowchart of 
Figure 6 in accordance with the present invention. 

Figure 14 is an illustration of an overlap detection of images in 
10 accordance with the present invention. 

Figure 15 is a flowchart of the overlap detection shown on Figure 

14. 

The present invention will be described in connection with a 
preferred embodiment; however, it will be understood that there is no intent to 
is limit the invention to the embodiments described. On the contrary, the intent is 
to cover all alternatives, modifications, and equivalents as may be included 
within the spirit and scope of the invention as defined by the appended claims. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

For a general understanding of the present invention, reference 
20 is made to the drawings. In the drawings, like reference numerals have been 
used throughout to designate identical elements. In describing the present 
invention, the following term(s) have been used in the description. 

^the-tefffH'da ta' 1 r e f e r s her ei n to phys i ca l s i gna l s that i ndjcatgjQ r 
include information. When an item of data canin^jcAte-<me^6Ta number of 
25 possible alternatives, the item of dataJras""one >f a number of "values." For 
^ example, a binary itempf-^JalaT also referred to as a "bit," has one of two 
values, interchangeably referred to as "1" and "0" or "ON" and "OFF" or "high" 
and "lowXA bit is an "inverse" of another bit if the two bits have different 
valued An N-bit item of data has one of 2N values. A "multi-bit" item of data is 
30 an^tem of data that iiiUIKJul. moru than one-blfT^ 
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"Memory circuitry" or "memory" is any circuitry that can store 
data, and may include local and remote memory and input/output devices. 
Examples include semiconductor ROMs, RAMs, and storage medium access 
devices with data storage media that they can access. A "memory cell" is 

5 memory circuitry that can store a single unit of data, such as a bit or other n- 
array digit or an analog value. 

A signal "indicates" or "selects" one of a set of alternatives if the 
signal causes the indicated one of the set of alternatives to occur. For 
example, a signal can indicate one bit set in a sequence of bit sets to be used 

10 in an operation, in which case the signal causes the indicated bit set to be 
used in the operation. 

An image may include characters, words, and text as well as 
other features such as graphics. A text may be included in a set of one or 
more images, such as in images of the pages of a document. An image may be 

15 processed so as to identify specific within the image, each of which is an 
image. 

A "bin" is a list of edge points that are approximately collinear. A 
list of edge points is converted into a sequence of bins that represent the 
boundaries as straight lines. 

20 An object may be of any size and shape and has "physical 

attributes" or characteristics including, but not limited, to position, shape and 
orientation. For example, an object may be a document, when the document is 
placed on the platen of an image input device and the region of the platen is 
digitized to produce a representation thereof. 

25 An item of data "defines" an image when the item of data includes 

sufficient information to produce the image. For example, a two-dimensional 
array can define all or any part of an image, with each item of data in the array 
providing a value indicating the color of a respective location of the image. 

An item of data "defines" an image set when the item of data 

30 includes sufficient information to produce all the images in the set. 
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Each location in an image may be called a "pixel." In an array 
defining an image in which each item of data provides a value, each value 
indicating the color of a location may be called a "pixel value". Each pixel 
value is a bit in a "binary form" of an image, a gray scale value in a "gray scale 

5 form" of an image, or a set of color space coordinates in a "color coordinate 
form" of an image, the binary form, gray scale form, and color coordinate form 
each being a two-dimensional array defining an image. 

An operation performs "image processing" when it operates on 
an item of data that relates to part of an image. 

10 Pixels are "neighbors" or "neighboring" within an image when 

there are no other pixels between them and they meet an appropriate criterion 
for neighboring. If the pixels are rectangular and appear in rows and columns 
within a two-dimensional image, each pixel may have 4 or 8 neighboring pixels, 
depending on the criterion used. 

15 An "edge" occurs in an image when two neighboring pixels have 

sufficiently different pixel values according to an appropriate criterion for the 
occurrence of an edge between them. The terms "edge pixel" or "boundary 
pixel" may be applied to one or both of two neighboring pixels between which 
an edge occurs. 

20 An "image characteristic" or "characteristic" is a measurable 

attribute of an image. An operation can "measure" a characteristic by 
producing data indicating the characteristic using data defining an image. A 
characteristic is measured "for an image" if the characteristic is measured in a 
manner that is likely to produce approximately the same result each time it 

25 occurs. 

A "version" of a first image is a second image produced using an 
item of data defining the first image. The second image may be identical to the 
first image, or it may be modified by loss of resolution, by changing the data 
defining the first image, or by other processes that results in modifying pixel 
30 values of the first image. 
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An "image input device" is a device that can receive an image 
and provide an item of data defining a version of the image. A "scanner" is an 
image input device that receives an image by a scanning operation, such as by 
scanning a document. A scanner may have a transparent surface (platen) or 

5 equivalent means to support a document during scanning. Other well-known 
image input devices include digital cameras, facsimile machines, and video 
recorders, or any image storage device having the capability to store data 
signals representative of the intensity of light reflected from the surface of 
objects at which the device is directed. A general purpose computer with an 

10 image creation program may also be an image input device. 

An "image output device" is a device that can receive an item of 
data defining an image and provide or render the image as output. A "display" 
is an image output device that provides the output image in human viewable 
form, and a "printer" is an image output device that renders the output image in 

15 a human viewable, hard copy form. 

Referring now to Figures 1 and 2, depicted therein is a digital 
printing system 20 in which the present invention finds particular use. 

For example, system 20 may be a Xerox DocuColor® 40 Digital 
Color Printing System or the Xerox 5775® Digital Color Copier. System 20 

20 includes a computer or data processing unit 22 (Figure 2) capable of receiving 
digital data representing an image of an original document 24 placed upon a 
platen of scanner 26. Computer 22 initially stores the digital input data from 
scanner 26 in memory 52 (e.g., RAM or magnetic disk storage) where the 
image may subsequently be accessed. In addition to the digital data, memory 

25 52 may also include program memory for the storage of object code suitable 
for directing the processor to execute image processing operations in 
accordance with the invention described herein. Computer 22 has associated 
therewith a user interface (U/l) 28 including one or more user input devices 30, 
such as a keyboard, a keypad, a mouse, trackball, stylus or equivalent pointing 

30 device, etc. 
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Also part of system 20 is a color image output device such as 
printer 34 which may include a laser-driven, xerographic printing engine as 
found in a number of commercially available printers. In a preferred 
embodiment, system 20 is employed to process the digital image data received 

5 as input from a scanner 26, utilizing image processing software running in 
processor 50, so as to produce an output file that may be rendered by printer 
34, stored in memory 50, and/or transmitted to another device via network 40. 
Although system 20 is depicted as an integrated unit in Figure 1, it will be 
appreciated that the system may also comprise a plurality of independent yet 

10 interconnected units. Referring to Figure 3, it will be appreciated that the 
document placed upon the scanner platen in system 20 may not be accurately 
registered along the registration edges 25. For example, a particular 
embodiment to which the following description will be directed is a single 
document object O placed in a central location on the platen of scanner 26 as 

15 illustrated in Figure 3. In accordance with the present invention it is desired to 
automatically identify the position, shape and rotation angle of object O. In a 
facsimile transmission system using a set of rollers to advance a document 
through a scanning nip, it will be appreciated that the document may become 
skewed during the digitization process and that the present application may 

20 have particular application to such systems as well. 

Given an input image generated by scanning the scanner platen, 
the present invention automatically identifies at least the position, shape and 
orientation angle of the object. As shown in the flow chart of Figure 4, the 
process carried out by computer 22 during the processing of the input image 

25 includes three general steps. First, at step 100 the object within the image is 
located and boundaries of the object are generally identified. Once the object 
is located, the shape of the object is modeled at step 200. Having located the 
object and modeled its shape, a structured image or similar digital document 
format representing the image and the object therein can be created as 

30 represented by step 300. The structured image preferably includes data 
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representing not only the image data itself, but data representing the location, 
shape or orientation of the object, or some combination thereof. Alternatively, 
the output may be a page description language format or equivalents formats 
suitable for storing the image information in a retrievable form. 
5 In an embodiment of the present invention, the scanned input 

image (or a lower resolution version thereof) is loaded into a memory frame 
buffer (RAM) where it is analyzed in accordance with the previously described 
steps. For purposes of the following detailed description, it is assumed that 
the object is distinguishable from the image background (e.g., the platen cover) 
10 and that the background of the image is contiguous. These simplifying 
assumptions are intended for purposes of explanation only and are not 
intended as limitations of the invention. One skilled in the art will appreciate 
that the invention described herein is extensible so as not to require operation 
only within the boundaries of such assumptions. 
15 *As dop i ctod by tho f l ow chart s, of Figures 5and^ 

location step 100 is performed by first identifying the backgroupd-rggion of the 
input image 102, characterizing the background regiorp-O^Tand then using the 
characteristic of the background region as a^s€ed, identifying all the pixels 
representing the background region wjth^an adaptive seed fill algorithm 106. 
20 Background pixels are pixels not^a^sociated with any objects, or more simply, 
they are pixels representatiy^of those regions lying outside of the objects, the 
values of which are controlled by the "background" against which the objects 
are placed during/scanning (e.g., the underside of the platen cover). One 
embodiment enpfploys the average color of a small region in the upper left-hand 
25 corner of the scanned image as an initial estimate of the background color. 
Alternatively, other sampling operations may be employed to determine the 
background color as described, for example, in US-A-5, 282,091 for a 
Programmable Apparatus for Determining Document Background Level by 
EktrglU^ 
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Once the background color is characterized at step 104, an 
adaptive algorithm is preferably applied to monitor the background color and to 
accurately identify the objects. An example of a seed fill algorithm suitable for 
use in the present invention is described in Graphics Gems I . A. Glassner Ed., 

5 Academic Press, pp. 275-277, 1990, hereby incorporated by reference. An 
adaptive algorithm is required because the background pixels may have 
significant color variation resulting from a variation in illumination over the 
platen area. The adaptive seed fill algorithm is applied to the scanned color 
image data using an initial seed point characterized by the background, for 

10 example, the upper-left corner of the image. Generally, the adaptive seed fill 
algorithm fills a binary frame buffer with a mask indicating all contiguous pixels 
identified as background pixels. In a simple embodiment, represented by step 
112, a pixel is considered to be a background pixel if its color falls within a 
small distance e of the current average background pixel value. This distance 

15 is calculated as an Euclidean metric in red, green, blue (RGB) color space 

d= SQRT ( (P r AdAvg r ) 2 +(P g -AdAvg g ) 2 +(P b -AdAvg b ) 2 ), 
where P k , AdAvg k are, respectively, the RGB components of the 
pixel under test and the average background value, and d is the distance 
measurement. The value of e is fixed and empirically determined in one 

20 embodiment. The test conducted at step 1 12 is: 

if d<e, then pixel P is a background pixel, else pixel P is a 

foreground pixel. 

The average background color is adaptively modified at step 1 14 
by taking the average value of the last N pixels that have been classified as 
25 background. For efficiency, the system preferably calculates the adaptive 
average using the equation: 

AdAvg' = (N*AdAvg-AdAvg+LastVal)/N, 

where AdAvg' is the modified average, AdAvg is the previous adaptive 
average, LastVal is the value of the last pixel identified as background, and N 
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is the averaging window. Clearly, this is not a true running average, but it 
tracks the running average adequately and is more computationally efficient 
than a strict running average calculation. Alternatively, the value of e can be 
adaptively modified. For example, e might be based on the standard deviation 

5 of the last several pixels identified as background, etc. 

It will be appreciated that alternative methods for the detection of 
background regions may be employed, and that the process of obtaining a 
binary map distinguishing the objects from the background may be 
accomplished using a simplified thresholding operation based upon the 

10 background color. In a preferred embodiment, the background color may be a 
function of the color of the platen cover, such as a white cover. As a further 
example, the digital color production system may employ a spectrally reflective 
or otherwise distinguishable platen cover that results in a background color 
that may be detected and distinguished from the objects. 

15 Having identified background pixels and created a representation 

of the object boundaries, noisy edges can be optionally smoothed using, for 
example, morphological filtering. Subsequently, contiguous foreground regions 
are located, step 122, thereby identifying the objects. Scanning the 
background mask generated by the adaptive seed fill operation (step 106) 

20 identifies objects. Starting with the upper left hand pixel, the mask is searched 
in a scan line fashion for a pixel not classified in the mask as a background 
pixel - thus identifying pixels associated with a foreground object. The use of 
the seed fill algorithm for identifying the background assures that foreground 
objects are closed, or in other words, a complete boundary is formed about the 

25 perimeter of the object. 

At step 124, tracing its edge identifies the boundary of an object. 
The boundary of the foreground object is traced using a simple 8-connected 
edge traversal operator which provides an ordered-set of points tracing the 
edge of the object. Such an edge traversal operation employs a contour tracing 

30 operation to generate a chain code in a manner similar to word or character 
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based recognition systems. R. Bozinovic et al. in "Off-Line Cursive Script 
Word Recognition", IEEE Transactions describes an 8-connected process, for 
example, on Pattern Analysis and Machine Intelligence, Vol. 1 1, No. 1 (January 
1989). In the situation of detecting multiple objects, one main object would 

5 preferably be selected using any of a number of techniques. For example, 
object selection could be accomplished by retaining the object with the largest 
perimeter or based upon the location of the object. 

Once the object has been located, as described with respect to 
step 100, the next general step, step 200, is to model the shape of the object. 

10 For purposes of simplicity, the following description will treat rectangular- 
shaped objects, however, it will be appreciated that the description of the 
invention is extensible to other polygons and even to shapes having portions 
thereof represented by curves (e.g., circular or elliptical objects). The result or 
output from step 100 is a set of bounding pixels or a set of edge traces 

15 preferably organized in the form of a linked list. These pixels or traces can be 
used to extract the object, but orientation is not yet determined. To improve 
the quality of the object extraction, the object traces are fitted to a model 
shape. Orientation information and other characteristics of the object may then 
be extracted from the fitted parameters. 

20 At step 400 of Figure 7, the 'bin' or result of step 100 i.e. the step 

of bounding pixels or set of edge traces in the form of a linked list is defined. At 
step 410, a reduction of the data set is necessary. The linked list defined at 
step 400 may be too large; in fact, it may be in the order of 1,000 data points or 
more. Therefore, it is necessary to reduce the amount of data of these 1 ,000 

25 or so data points. Using a factor N divisible into the total data points does 
such data reduction is performed. Typically the factor is around the value of 5, 
or may be another factor that enables a 100-200 point's border. For example, 
if there were 1,000 data points, to reduce such a data set to within 100-200 
points would require a factor between 5 and 10. The data set is then reduced 

30 to a set of 100-200 points. 
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•Aftei 1Mb data mduci l on, at step 4^ !0 6t hTgureJi^aft-est^^ 
the angle of the line passing through ea^^enrrSining point on the contour is 
preformed. As shown in Figm^^T^Tmodified linear regression in a particular 
window (W) centere^orfeach point is performed so as to estimate an angle of 
the line passiijginrough each remaining point of the contour determined by the 
set of d^ta points. Initially a modified linear regression is done on a small 
window centered on a point (A) where each linear regression requires a series 

lations. 



icJcfitions, multiplication's, anc 

trh- particu l ar, a st a ndard rpgrps si on algorithm fits data to i thgJift Q 
y=mx+k. However, using a standard regression algorithmjo-p these 
functions can lead to erratic results as the lin^Sipproaches a vertical. 
Therefore, in the present invention, the algorithm is modified so as to account 
for these erratic results. As shoyjtnHfiFigure 8 in each of the four quadrants 
indicated therein, a stancjartfregression is used for the lines that are more 
horizontal in two guaclrants with a slope between -45° and +45°. When the 
slope is noUFrlhe identified degrees, that is, when the lines are more vertical 
with a / £kJpe greater than 45° and in the other two quadrants, an inverted linear 
repression based on the inverted linear equations x=1/ny-k/m is performed. 

^following equation^ 



Tope angle is deterr 

I avg (xy) - avg(x)avg(y) l> lavg(x 2 )- avg(x) 2 1 when the slope is more 
vertical 

I avg(xy) - avg(x)avg(y)l < lavg(x 2 )- avg(x 2 )l when the slope is more 
horizontal 
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f Wn thn -^l np n n .nl ru ilatinn arp flnro m plishftrl ^ 

figure 6, each point associatedjwitl]^^ by performing a 

bin categorizing o^atttjfftogenerate a series of bins. For example, as shown 
in Figure 9^-blns B1, B2, B3, and B4... are generated from a series of angles, 
which jre associated with each p o int Th Q r\\yp< $ n f c f np A?n \r^ 0 rPi \^ r \ 7 ^ 
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Tjrgups of adjacent boundary points thatshajg^ i.e. ouhVei t 

list of boundaiTjiomteififo^ of bins (B1, B2, B3...) where each of 

the bips^onsists of a set of collinear points so as to generate a boundary of an 
=tge made Up of a set of straight line segmeRtSr--^* 
5 When generating the above-described line segments, boundary 

point angles of the predecessor points may differ from the angle of the current 
point by a substantial amount. If, for example, the boundary point 1 angle differs 
from the previous point by less than a particular amount E, then the point is 
added to the current bin. If the boundary point angle differs by more than a 
10 particular amount E, then it is assigned to a new bin. Each bin (B1, B2, B3...) 
represents a set of collinear points along the edge. 

Once the bin categorization is completed and the each point is 
assigned with the appropriate bin some of the bins contain categorized contain 
to few data points. Therefore at step 440 of figure 7 it is necessary to remove 
15 the bins which contain few or less than a minimum number of data points. 
Typically this removal is performed when there are approximately 5 data 
points. As an example of this removal in Figure 10, the smaller bins A, B, and 
C are removed because they contain less than 5 data points. 

After the bin removal of the smaller bins, an average angle of 
20 points within each of the remaining bins is determined at step 450. The same 
modified regressions described above are now applied to the remaining bins 
generating an average slope and average slope angle calculation for each of 
the remaining bins. 

Next, the remaining bins that share common angles are 
25 combined at step 460. These remaining bins were previously not adjacent to 
each other but became adjacent at step 440 by the removal of small bins found 
in between the non-adjacent bins. When these bins in between are removed, 
the bins that are left and that are now adjacent can sometimes be collinear. 
Therefore, the angle of each bin is checked and if adjacent bins are collinear, 
30 the bins are joined into a single bin at step 460. Such a procedure is 
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illustrated in Figure 11, were a bin defined by AL-AR and another bin defined 
by BL-BR are compared to each other. The midpoint between each point 
AM/BM is calculated and the slope of the segment AB is determined and 
compared to the slopes of each of the bins. If the slopes are within 10°, an 
5 amount designated as E, then the following calculation is performed. 

(Slope (AB) - Slope (A)) < E & (Slope(AB) - Slope(B)) < E where 
E is approximately 10 degrees. There are instances when the slope 
calculations are susceptible to noise in short bins, hence the value is modified 
if one of the bins is determined to be very short with respect to the other bin. 
io Finally, at step 470, the intersection between each bin and the 

next bin is calculated. The midpoint x, y, and slope angle M of each bid is 
used to calculate the intersection xy between the two bins, 
ki = y 1 - miX! 
k 2 = y2 - m 2 X2 
15 x = k 2 -k 1 /iTh - m 2 

y = rrhx + k t 

The angle of intersection is calculated from the vector cross 
product between vectors pointing from the intersection point to the midpoints: 
A x B = [(x1-x)(y2-y) - (x2-x)(y1-y)] k 
20 = [A] [B] sin (9) k 

6 = sin" 1 {(x1-x)(y2-y)-(x2-x)(y1-y) / 

At this stage of the procedure the edge boundary has been 
reduced to a small number of bins. The next step is to apply a recognition 
algorithm because the final number of bins usually is 4 for rectangular images. 
25 However, there are cases when there are more than 4 rectangular images and 
so it is necessary to apply a recognition algorithm. This results because smart 
platens sometimes have regions that are always easily distinguished from the 
background images and the boundaries of the detected images bleed into the 
background images. Because of the bleeding, the detected boundaries do not 
30 match the actual images in certain locations. Therefore, the boundaries fail to 
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fit the actual image and it is necessary to apply further procedures to make the 
boundaries fit particular images. 

At step 500 of figure 12, the list of bins which is generated is 
browsed to identify bins that have 90° intersections with neighboring bins or 

5 the bins that approximate 90° angles with their neighbors. If there are bins that 
are ascertained to be at 90° angles, then these bins are overlooked because 
often the bins that have these angles are from images misaligned against the 
edges of the platen. Therefore, if the bin is found to be a right angle then the 
bin is more than likely generated from images aligned at platen edge and 

10 accordingly should be discarded. Moreover, it is necessary to search for all 
right or 90° angles because these angles also may correspond to the corners 
of an image. To be considered a right angle a bin has to lie between 85-95°. 
As indicated earlier, in step 470 of Figure 7 the angle of intersection is 
calculated and is used in step 500 to determine whether or not there are right 

15 angles. 

Next, as illustrated in Figure 12, at step 530, once the right angle 
is identified, the flow proceeds to determine whether a rectangle can be 
constructed and can be test fitted to the boundary. To construct a rectangle 
the following steps shown in the flow chart of Figure 13 are performed. 

20 At step 700 of figure 13, an estimation of the rotation angle of a 

rectangle as the slope angle of the right angle of the bin is performed. Next, at 
step 710 all the points on the boundary of the rectangle are rotated according 
to the rotation angle found in step 700. At step 720, the axis aligned bounding 
box is calculated. Then the corner coordinates of the axis aligned bounding 

25 box are un rotated at step 730. The flow then proceeds to step 540 of figure 
12. 

At step 540 of figure 12, a test of the similarity measure or 
similarity fit is performed by counting the number of boundary points in each of 
the two bins that are at right angles and that lie within a measure of 3 pixels of 
30 the resulting bounding box divided by the total number of points in the bin. If 
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the similarity measure is less than .9 the rectangle test is considered to be a 
good match and the flow proceeds until done. If the similarity test is less than 
.9 no results are found the flow proceeds to step 500 for the next pair of bins 
at step 545 and is repeated for the next right angle pair of bins. If no angles 
5 have similarity then the flow proceeds to step 550 to an after final search until 
no bins are left. 

Next, at step 560 the longest bin is searched and the flow 
proceeds to determine if a rectangle can be constructed to be test fitted to the 
boundary. In order to construct the rectangle the following steps are 

10 necessary as shown in Figure 13. Initially, an estimation of the rotation or 
angle is determined at step 700, then all the points are rotated at step 710, a 
calculation of an axis aligned bounding box at step 720 and the corner 
coordinates of the axis-aligned bounding box are unrotated. The flow then 
proceeds to step 580. There a rectangular test fit is performed and is 

15 determined to be good if the test fit similarity measure is greater than .9 
resulting in a good match to be determined. If the similarity measure is less 
than .9 the flow proceeds back to step 560 and is repeated again for the next 
longest bin. If the test fit similarity test fails again then the step proceeds to the 
after final search with no bins left Step 600. The flow then proceeds to Step 

20 610 where the longest bin is searched. Then a rectangle is constructed at step 
620 from the longest bin. 

At this point, a bounding box has been identified along with its 
rotation angle. The next step is to eliminate those boundaries that are 
determined not to actual boundaries of the bounding box. If the aspect ratio of 

25 the bounding rectangle is more than 15:1 or less than 0.075:1 then a boundary 
is considered not to be conforming as a boundary of the bounding box and is 
rejected. Also, if the actual width or height is less than 5 pixels; the boundary 
is again considered not to be desirable and is rejected. 

Finally, it is possible that a single image may show up as two or 

30 more boundaries if the bleeding is excessive. In this case, a containment 
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algorithm is applied to test if a boundary is contained within another. Each 
boundary is tested against all other boundaries to see if there is a 25% or more 
overlap between the two boundaries. If such an overlap occurs, the two 
boundaries are joined and the bounding rectangle is recalculated (go to step 
5 500). 

The points that make up a boundary are generally the verticals of 
a many-sided polygon. The containment test as shown in figure 14, is a 
procedure to calculate the overlap between two polygons P1 and P2. 

Initially, as shown in figure 15, a determination of the minimum 
io and maximum P1 & P2 y-values (step 800) for each polygon is performed. In 
the case of Smart Platen, these values are already known and do not have to 
be recalculated. Next, for each polygon, there is an allocation of an array large 
enough to store two integers for all possible values of y in the polygon (step 
810). For example, if y were to have a range of 332, then it is necessary to 
15 allocate an array of 332 pointers to two integers. Third, for each polygon, X 
values are determined by scanning through the vertices and determining the 
min and max values of X for each Y-value (step 820). However, vertices may 
not exist for each y-value, therefore Y-values may have to be interpolated. As 
an example, consider two vertices given by (x1,y1) and (x2,y2) where y2-y1=K. 
20 It is necessary to interpolate K-1 values of X, one for each value of y. The 
algorithm for interpolating x is as follows: 
(Step 830) 
dx = (x2-x1)/K; 
for 0=0; j<K; ++j) 
25 { 

x = x1 + j*dx; 

} 
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Once the arrays containing the min and max x-values for each 
polygon is created at step 810, an overlap measurement determination is 
performed at step 840. 

The test for vertical overlap is if the least y-value is greater than 
5 the largest y-value step 840, there is no overlap and then the flow proceeds to 
step 880. Also, if the largest y-value of P, is less than the least Y-value of P2, 
then there is no overlap, the flow proceeds to step 800. However, for those y- 
values where both P1 and P2 overlap, the horizontal overlap is summed and 
the following checks are performed. First, if the minimum x-value of P1 is 
o greater than the maximum x-value of P2 then the flow proceeds to step 860. 
Second, if the maximum x-value of P1 is less than the minimum x-value of P2, 
then the flow proceeds to step 870 and an estimation of the overlap for that y- 
value is performed as follows: 

QveilafT+ - Max i mum P l xmm, P2xmin) - Min i mum (P l xmax, P 2 xmax) + 1 (atop - 

This determination shown above returns the overlap between two 
polygons. When applied to the inventions platen, we normalize the returned 
overlap area by dividing by the smaller of the two borders. If the result 
exceeds 25%, the smaller boundary is considered to be an orphan or a part of 
20 the larger boundary and the two are merged and a new bounding rectangle is 
calculated. 

By performing the above steps a good bounding rectangle is 
identified. Further techniques such as file extraction are also applicable to 
further process the received data. Other kinds of processing steps may also 
25 be performed for detecting non-rectangular objects, overlapping rectangular 
objects, etc. 

While the invention is described with reference to a particular 
embodiment, this particular embodiment is intended to be illustrative, not 
limiting. Various modifications may be made without departing from the spirit 
30 and scope of the invention as defined in the amended claims. Modifications 
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and alterations will occur to others upon reading and understanding this 
specification, therefore it is intended that all such modifications and alterations 
are included insofar as they come within the scope of the appended claims or 
equivalents thereof. 
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